3 research outputs found
SKoPe3D: A Synthetic Dataset for Vehicle Keypoint Perception in 3D from Traffic Monitoring Cameras
Intelligent transportation systems (ITS) have revolutionized modern road
infrastructure, providing essential functionalities such as traffic monitoring,
road safety assessment, congestion reduction, and law enforcement. Effective
vehicle detection and accurate vehicle pose estimation are crucial for ITS,
particularly using monocular cameras installed on the road infrastructure. One
fundamental challenge in vision-based vehicle monitoring is keypoint detection,
which involves identifying and localizing specific points on vehicles (such as
headlights, wheels, taillights, etc.). However, this task is complicated by
vehicle model and shape variations, occlusion, weather, and lighting
conditions. Furthermore, existing traffic perception datasets for keypoint
detection predominantly focus on frontal views from ego vehicle-mounted
sensors, limiting their usability in traffic monitoring. To address these
issues, we propose SKoPe3D, a unique synthetic vehicle keypoint dataset
generated using the CARLA simulator from a roadside perspective. This
comprehensive dataset includes generated images with bounding boxes, tracking
IDs, and 33 keypoints for each vehicle. Spanning over 25k images across 28
scenes, SKoPe3D contains over 150k vehicle instances and 4.9 million keypoints.
To demonstrate its utility, we trained a keypoint R-CNN model on our dataset as
a baseline and conducted a thorough evaluation. Our experiments highlight the
dataset's applicability and the potential for knowledge transfer between
synthetic and real-world data. By leveraging the SKoPe3D dataset, researchers
and practitioners can overcome the limitations of existing datasets, enabling
advancements in vehicle keypoint detection for ITS.Comment: Accepted to IEEE ITSC 202
An Open-Source Platform for Human Pose Estimation and Tracking Using a Heterogeneous Multi-Sensor System
Human pose estimation and tracking in real-time from multi-sensor systems is essential for many applications. Combining multiple heterogeneous sensors increases opportunities to improve human motion tracking. Using only a single sensor type, e.g., inertial sensors, human pose estimation accuracy is affected by sensor drift over longer periods. This paper proposes a human motion tracking system using lidar and inertial sensors to estimate 3D human pose in real-time. Human motion tracking includes human detection and estimation of height, skeletal parameters, position, and orientation by fusing lidar and inertial sensor data. Finally, the estimated data are reconstructed on a virtual 3D avatar. The proposed human pose tracking system was developed using open-source platform APIs. Experimental results verified the proposed human position tracking accuracy in real-time and were in good agreement with current multi-sensor systems
Fusion of Multiple Lidars and Inertial Sensors for the Real-Time Pose Tracking of Human Motion
Today, enhancement in sensing technology enables the use of multiple sensors to track human motion/activity precisely. Tracking human motion has various applications, such as fitness training, healthcare, rehabilitation, human-computer interaction, virtual reality, and activity recognition. Therefore, the fusion of multiple sensors creates new opportunities to develop and improve an existing system. This paper proposes a pose-tracking system by fusing multiple three-dimensional (3D) light detection and ranging (lidar) and inertial measurement unit (IMU) sensors. The initial step estimates the human skeletal parameters proportional to the target user’s height by extracting the point cloud from lidars. Next, IMUs are used to capture the orientation of each skeleton segment and estimate the respective joint positions. In the final stage, the displacement drift in the position is corrected by fusing the data from both sensors in real time. The installation setup is relatively effortless, flexible for sensor locations, and delivers results comparable to the state-of-the-art pose-tracking system. We evaluated the proposed system regarding its accuracy in the user’s height estimation, full-body joint position estimation, and reconstruction of the 3D avatar. We used a publicly available dataset for the experimental evaluation wherever possible. The results reveal that the accuracy of height and the position estimation is well within an acceptable range of ±3–5 cm. The reconstruction of the motion based on the publicly available dataset and our data is precise and realistic